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Abstract 


For NASA’s air transportation research program, we demonstrate an approach to 
integrating reliability, performance, and operational procedure modeling into a 
system safety analysis. Our methodology is distinguished by its ability to merge 
system design/functionality information with the dynamic parameterization of a 
system’s situation to generate accident statistics and measures of reliable system 
operation. In addition, this approach can be employed to perform sensitivity 
analyses to identify weak points in the system’s operation and design. 

Our approach to system safety analysis results from the integration of a Reliability 
model and an Interaction-Response model. The Interaction-Response model pro- 
vides information regarding the frequency of encounters and the predicted out- 
come of those encounters as a function of the system’s alerting system and ability 
to resolve encounters. The Reliability model provides, as a function of time, prob- 
abilities associated with the critical systems’ availability and failure states. Scaling 
the conditional operational safety metrics provided by the Interaction-Response 
Model by the system state probabilities produced by the Reliability model creates 
the system-level safety statistics. 

Products of this analysis include 

♦ predicted incident (encounter) statistics; 

♦ predicted accident statistics; and 

♦ predicted false alarm statistics, as well as system availability and reliabil- 
ity. 

As an application of this methodology, we have considered the problem of simul- 
taneous independent approaches of two aircraft on parallel runways (independent 
approaches on parallel runways). An illustration of how our approach can be ap- 
plied for system sensitivity analysis is also given. 
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Chapter 1 

Introduction and Summary 


Problem Definition 

The continuing growth of air traffic will place demands on NASA’s Air Traffic 
Management (ATM) system that cannot be accommodated without the creation of 
significant delays and economic impacts. To deal with this situation, work has be- 
gun to develop new approaches to providing a safe and economical air transporta- 
tion infrastructure. Many of these emerging air transport technologies will 
represent radically new approaches to ATM, both for ground and air operations. 

The essential questions that must be answered before adopting a new approach to 
air transport management are as follows: 

♦ Is the new system safe? 

♦ What are the costs of implementing the new system? 

♦ What are the direct economic benefits of the new system with respect to 
reduced delays or reduced airline costs? 

♦ What are the indirect economic benefits of the new system with respect to 
deferred construction of new airports? 

♦ What is the optimal transitioning process from the current system to the 
new system to ensure safety? 

To answer these questions and thus select a viable ATM concept, analysis will 
contain 

♦ performance models to measure delays, throughput, and aircraft density; 

♦ safety models to measure aircraft interactions and predict accident statis- 
tics; and 

♦ economic models to measure system costs and associated benefits. 

As shown in Figure 1-1, each of these three classes of analysis models rely on the 
others for some of their inputs. In other words, the design, analysis, and evaluation 
of Air Traffic Management concepts must be treated as an interactive process in 
which the analyses provide crucial feedback to system developers, as well as the 
benefits and safety metrics required to support program advocacy. 
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Figure 1-1. Integrated System Analysis and Development 
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Thus, the primary focus in developing a methodology for integrated system analy- 
sis must be to understand and model the interactions among performance models, 
safety models, and economic models. By doing so, the methodology can be used 
to 


♦ identify the drivers or weak links in the current system; 

♦ provide guidance in selecting topics for improvement studies; 

♦ measure net improvement in a proposed concept, distinguishing candidate 
concepts that represent global gains from those that solve one problem by 
creating another; and 

♦ provide a foundation for cost/benefit analyses that can measure true sys- 
tem-wide impacts. 

Products of this analysis include 

♦ predicted incident (encounter) statistics; 

♦ predicted accident statistics; and 

♦ predicted false alarm statistics, as well as system availability and reliabil- 
ity. 

As an application of this methodology, we have considered the problem of simul- 
taneous independent approaches of two aircraft on parallel runways (independent 
approaches on parallel runways [IAPR]). 
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Integrated System Safety Analysis: Concept, 
Approach, and Products 


We develop and demonstrate an integrated safety analysis methodology, one of 
the key elements of an integrated system analysis capability. This methodology is 
distinguished by its ability to merge system design/functionality information with 
the dynamic parameterization of a system’s situation to measure accident statistics 
and reliable system operation. The “system” may include both air and ground sub- 
systems within this analysis framework. In addition, it can perform sensitivity 
analyses to identify weak points in the system’s operation and design. This is il- 
lustrated in Figure 1-2. 

Figure 1-2. Integrated Safety and Reliability Modeling and Evaluation 



• Infrastructure “ilities" metrics 

* Operations safety metrics 


On the left side of Figure 1 -2 are the steps leading from requirements derived for 
an operational concept to the development of a Reliability Model of the system 
architecture, which has been proposed to meet those requirements. This represents 
a traditional reliability/safety modeling process. On the right are the models re- 
quired to capture the environment in which the system is to operate, as well as the 
interaction of those environmental models with response models representing the 
execution of the rules and procedures that have been developed for the candidate 
concept. This represents a modeling process for the dynamic analysis of the sys- 
tem’ s situation. 
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Our approach to system safety analysis results from the integration of the Reli- 
ability Model and the Interaction-Response Model. The Interaction-Response 
Model provides information regarding the frequency of encounters and the pre- 
dicted outcome of those encounters as a function of the system’s alerting system 
and ability to resolve encounters. The Reliability Model provides, as a function of 
time, probabilities associated with the critical systems’ availability and failure 
states. Scaling the operations safety metrics from the Interaction-Response Model 
by the system state probabilities from the Reliability Model creates the system- 
level safety statistics. This process is illustrated in Figure 1-3. 


Figure 1-3. Combining Model Outputs 
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Products of this analysis include 

♦ predicted accident statistics, 

♦ predicted false alarm statistics, and 

♦ predicted system availability and reliability. 

Moreover, as the operational concept evolves, the impact of changes in system 
architecture, rules and procedures, and operational scenarios can be easily re- 
evaluated with this methodology. 

Figure 1-2 makes it clear that system safety is being addressed from a variety of 
perspectives, each of which affects safety. These include 

♦ system functionality, the analysis of how reliably the system components 
perform; 

♦ rules and procedures, the analysis of how the system is designed to re- 
spond in both safe and unsafe situations; and 

♦ operational scenario, the analysis of the environment in which the system 
is expected to operate. 
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Integrating models that quantify each one of these three elements creates an analy- 
sis capability that is now system-wide and responsive to ongoing changes in the 
definition and requirements of the operational concept. 

Application to Independent Approaches on 
Parallel Runways 

As an application of this methodology, we have considered the problem of simul- 
taneous, but independent approaches of two aircraft on parallel runways (i.e., 
IAPR). In visual meteorological conditions (VMC), the pilots may accept respon- 
sibility for maintaining separation between their aircraft by visual means. For ap- 
proaches conducted during instrument meteorological conditions (IMC), air traffic 
control personnel are responsible for the separation between the aircraft. The Fed- 
eral Aviation Administration (FA A) allows independent parallel approaches to be 
carried out in VMC with a runway separation minimum of 700 feet. In IMC, inde- 
pendent approaches may be conducted to runways spaced at least 4,300 feet apart. 
This minimum is reduced to 3,400 feet if the airport is equipped with the Preci- 
sion Runway Monitor (PRM) system. 

A study performed by the Boeing Commercial Airplane Group has predicted sig- 
nificant increases in runway capacity per hour if dependent approaches could be 
replaced by independent approaches. Because of capacity increases to be gained, it 
is desirable to reduce the minimum runway separation required for independent 
approaches. 

A variety of projects have been undertaken within the past several years to explore 
alerting systems and cockpit displays for the parallel approach situation. Aircraft 
are more closely spaced during parallel approach than during any other phase of 
flight. The potential exists for an aircraft on either runway to deviate off course 
toward another aircraft on the parallel runway. To increase safety, an alerting sys- 
tem is used to warn flight crews of these blundering aircraft. The goal of the 
alerting system is to ensure adequate separation between aircraft while allowing 
parallel approaches to be carried out safely. With reference to our integrated safety 
model in Figure 1-2, these studies represent Interaction-Response Models. 

Independent Approaches on Parallel Runways Concept 
and Operational Procedures 

Figure 1-4 illustrates the elements of a typical IAPR concept. 
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Figure 1-4. Parallel Runway Concept 



The IAPR system takes advantage of advances that have been made in communi- 
cation, navigation, and surveillance technologies. Primary among these is GPS- 
based navigation and digital communications for both surveillance and pilot in- 
formation exchanges (ADS-B). GPS-based navigation, with appropriate augmen- 
tation when needed, will provide much more accurate aircraft position and 
velocity information, reducing the need for large protective bubbles around air- 
craft. The accuracy and speed of the ADS-B surveillance data link system is also 
key to successful implementation of the IAPR concept. 

The assumed operational procedure for the IAPR addressed here is as follows: 

♦ On-board GPS system provides accurate, timely positional information of 
own ship. 

♦ Position of own ship is broadcast via ADS-B. 

♦ Positions of other ships are received and processed via ADS-B. 

♦ Location of own ship relative to runway approach and other ships is proc- 
essed and displayed on a cockpit display of traffic information (CDTI) 
monitor. 
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♦ Alerting logic sounds alert according to levels of “encounter” criteria an- 
ticipated. 

♦ Avoidance maneuver is initiated in order to avoid “near collision” event. 

Lacking any involvement of ground control, the LAPR concept just described rep- 
resents a severe and possibly worst-case scenario. It is, however, more manage- 
able from a modeling standpoint for this first application. Certainly, future work 
must include models for ground control interaction with aircraft. 

Independent Approaches on Parallel Runways Analysis 
Framework Overview 

Figure 1-5 illustrates the IAPR analysis framework organized with respect to four 
major components: system Reliability Model, Impact Model, Interaction- 
Response Model, and derivation of system safety statistics. 


Figure 1 -5. IAPR Analysis Framework 



Note: RNP = required navigational performance. 


Compared to Figures 1-2 and 1-3, the new feature in Figure 1-5 is the “Impact 
Model.” The function of the Impact Model is to associate each system functional 
state employed in the Reliability Model with an operational capability of the air- 
craft and pilot. For example, a fully operational aircraft can execute a normal ap- 
proach. The system functional state, fully operational, is associated with the flight 
capability, normal approach. Furthermore, the likelihood of the system functional 
state, fully operational, is quantified by the Reliability Model, while (conditional) 
safety metrics for the normal approach are determined from the Interaction- 
Response Model through a simulation process. The interaction-response simula- 
tion model includes a specific example of the alerting logic currently under inves- 
tigation by NASA. The resulting system-level safety statistics are calculated by 
scaling the conditional safety metrics with the likelihood of the system functional 
state as illustrated in Figure 1-3. 
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An in-depth examination of each analysis component is presented in Chapter 3. 
The final results are summarized here in Tables 1-1 and 1-2. 

Table 1-1. Combined Results at 1,700-Foot Runway Spacing 

System safety statistic (t) = £ Pr(simulation safety stat.l flight track) x Pr (flight track)(t) 

Flight tracks 


Flight tracks 

Conditional simulation safety statistics 
Rel. op. Collisions False alarms 

Probability flight trk 

t = 4hrs. t = 10 hrs 

[norm_145, norm_145] 

1 

0 

0 

9.99e-1 

9.98e-1 

[norm_145, fake_145] 

.9544 

0 

.0456 

3.65e-6 

9.1 e-6 

[norm_145, oadj_145] 

.9125 

0 

.0875 

3.65e-6 

9.1 e-6 

[norm_145, sb5_145] 

.996 

0 

.0040 

1.72e-4 

4.3e-4 

[normal 45, sh5_145] 

.9854 

.0092 

.0054 

1.726-4 

4.3e-4 

[normal 45, slo 145] 

.9872 

.0091 

.0037 

1.726*4 

4.36-4 

[norm 145, bl15_145] 

.996 

.0018 

.0022 

7.15e-6 

1.8e-5 

[norm_145, b!30_145] 

.9872 

.0037 

.0091 

7.15e-6 

1 .80e-5 


System safety statistics 

Rel. op. (4) = 

0.9995 

Collisions (4) = 

3.19E-6 

False alarms (4) = 

2.82E-6 

Rel.op. (10) 

0.9993 

Collisions (10) = 

7.97E-6 

False alarms (10) = 

7.05E-6 


The Reliability Model was evaluated for both 4 and 10 hours of flight prior to the 
aircraft beginning the runway approach. System safety statistics are computed for 
each time period and reflect the fact that as the time in flight increases prior to 
runway approach, the overall hazard increases and reliable operation decreases. 

In addition to the 1,700-foot spacing, we completed a baseline evaluation at both 
2,500-foot and 3,400-foot runway spacing. The three sets of safety statistics are 
given in Table 1-2. 

Table 1-2. Safety Statistics at 1,700-Foot, 2,500-Foot, and 3,400-Foot 

Runway Spacings 

2,500-fool spacing 

System safety statistics 

Ral. op. (4) = 0.999524 

Collisions (4) = 3.160E-6 

False alarms (4) = 1.017E-6 


Rel.op. (10) = 0.999310 

Collisions (10) = 7.901 E-6 

False alarms (10) = 2.544E-6 


3,400-foot spacing 


System safety statistics 

Rel. op. (4) 

= 0.999535 

Collisions (4) 

= 7.179E-7 

False alarms (4) 

= 1.013E-6 

Rel. op. (10) 

= 0.999339 

Collisions (10) 

= 1 .796 E-6 

False alarmB (10) 

= 2 533 E-6 


1 ,700-foot spacing 


System safety statistic* 

Rel. op. (4) 

= 0 999531 

Collisions (4) 

= 3.187E-6 

False alarms (4) 

= 2.81 9E-6 

Rel. op. (10) 

= 0.999329 

Collisions (10) 

= 7. 968 E-6 

False alarms (10) 

= 7. 047 E-6 


As the runway spacing changes, only the conditional safety statistics change in 
response; the scaling probabilities from the Markov model remain the same. The 
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actual numerical values should be considered hypothetical and devised for the 
purposes of this example; nevertheless, the trend of the data is reasonable and 
what one would expect. As the time in flight increases prior to runway approach, 
the overall hazard increases and reliable operation decreases. As the runway 
spacing between aircraft increases, the probabilities of collision and false alarm 
decrease while reliable operation increases. 

In order to demonstrate the approach, we have employed simple models. How- 
ever, the approach is one wherein models can be appropriately tailored for the 
level of detail available or desired. 

We conclude with an example of sensitivity analysis to show how this safety 
methodology can be used to suggest and evaluate design changes leading to im- 
proved system performance. 

Sensitivity Analysis: An Example 


The results of the integrated safety analysis can be used to determine how sensi- 
tive the safety statistics are to features of the system architecture, rules, and oper- 
ating procedures, or operational scenarios and environment. By understanding 
these sensitivities, design improvements can be proposed and evaluated with a 
cost/benefit tradeoff analysis. But the first step is to isolate the sensitivity. 

Referring back to Table 1-1, Combined Results at 1,700-Foot Runway Spacing, 
observe that the slow heading change blunders of 5 and 10 degrees have the high- 
est collision probabilities: 0.0092, and 0.0091, respectively. In addition, these 
tracks have the largest probabilities of occurrence with a value of 1.72E-4 at 
4 hours and 4.3E-04 at 10 hours. In our example, these two tracks are associated, 
in part, with a degraded navigation capability such as a faulty INS subsystem. 

Suppose it were possible to acquire a new, upgraded Inertial Navigation System 
(INS) component with a failure rate reduced from IE-04 down to IE-05. Replac- 
ing the “old” INS component by the new, an improved element would result in 
reduced probabilities of occurrence for the slow 5 and 10 degree heading blun- 
ders, namely, 5.3E-5 at 4 hours and 1.32E-4 at 10 hours. 

Reevaluating the system statistics now yields improvements in collision and false 
alarm probabilities as shown in Table 1-3. 
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Table 1-3. Comparison of Results for Improved INS 


Original INS 

Improved INS 

Collisions (4) = 3. 19E-06 
False alarms (4) = 2.82E-06 
Collisions (10) = 7.97E-06 
False alarms (10) = 7.05E-06 

Collisions (4) = 1 .01 E-06 
False alarms (4) = 1 .02E-06 
Collisions (10) = 2.515E-06 
False alarms (10) = 2.54E-06 


Note: Numbers in parentheses denote length of flight in hours. 


Alternatively, a rules and procedures change could be made whereby independent 
parallel landings would be precluded when the aircraft is in the degraded naviga- 
tion state. Costs and benefits would have to be evaluated for both the architecture 
option and rules/procedures option to arrive at the best course of action to im- 
prove the overall system performance and reduce the liability of accident and false 
alarm. In either case, the integrated safety analysis can be exercised interactively 
and iteratively to arrive at the best solution. 

Summary 

We have demonstrated an approach to integrating reliability, performance, and 
operational procedures modeling into a system safety analysis. Our methodology 
is distinguished by its ability to merge system design/functionality information 
with the dynamic parameterization of a system’s situation in order to measure ac- 
cident statistics and reliable system operation. In addition, it can perform sensitiv- 
ity analyses to identify weak points in the system’s operation and design. 

Our approach to system safety analysis results from the integration of the Reli- 
ability Model and the Interaction-Response Model. The Interaction-Response 
Model provides information regarding the frequency of encounters and the pre- 
dicted outcome of those encounters as a function of the system’ s alerting system 
and ability to resolve encounters. The Reliability Model provides, as a function of 
time, probabilities associated with the critical systems’ availability and failure 
states. Scaling the operations safety metrics from the Interaction-Response Model 
by the system state probabilities from the Reliability Model creates the system- 
level safety statistics. Products of this analysis include 

♦ predicted incident (encounter) statistics; 

♦ predicted accident statistics; and 

♦ predicted false alarm statistics, as well as system availability and reliabil- 
ity- 
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Chapter 2 

Integrated Safety Analysis Overview 


Concept, Approach, and Products 

In this report, we develop and demonstrate an integrated safety analysis method- 
ology, one of the key elements of an integrated system analysis capability. This 
methodology is distinguished by its ability to merge system design/functionality 
information with the dynamic parameterization of a system’s situation in order to 
measure accident statistics and reliable system operation. The “system” may in- 
clude both air and ground subsystems within this analysis framework. In addition, 
it can perform sensitivity analyses to identify weak points in the system’s opera- 
tion and design. This is illustrated in Figure 2-1. 

Figure 2-1. Integrated Safety and Reliability Modeling and Evaluation 



• Infrastructure “ ilities " metrics 

• Operations safety metrics 


On the left side of Figure 2-1 are the steps leading from requirements derived for 
an operational concept to the development of a reliability model of the system ar- 
chitecture, which has been proposed to meet those requirements. This represents a 
traditional reliability/safety modeling process. On the right are the models re- 
quired to capture the environment in which the system is to operate, as well as the 
interaction of those environmental models with response models representing the 
execution of the rules/procedures that have been developed for the candidate con- 
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cept. This represents a modeling process for the dynamic analysis of the system’s 
situation. 

Our approach to system safety analysis results from the integration of the Reli- 
ability Model and the Interaction-Response Model. The Interaction-Response 
Model provides information regarding the frequency of encounters and the pre- 
dicted outcome of those encounters as a function of the system’s alerting system 
and ability to resolve encounters. The Reliability Model provides, as a function of 
time, probabilities associated with the critical systems’ availability and failure 
states. Scaling the operations safety metrics from the Interaction-Response Model 
by the system-state probabilities from the Reliability Model creates the system- 
level safety statistics. This process is illustrated in Figure 2-2. 


Figure 2-2. Combining Model Outputs 
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Performance metrics from 
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Weighted system 
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Products of this analysis include predicted accident statistics, predicted false alarm 
statistics, and predicted system availability and reliability. Moreover, as the op- 
erational concept evolves, the impact of changes in system architecture, rules and 
procedures, and operational scenarios can be easily accounted for with this meth- 
odology. 

From Figure 2-1, it is clear that system safety is being addressed from a variety of 
perspectives, each of which impacts safety. These perspectives include (1) system 
functionality, the analysis of how reliably the system components perform; 

(2) rules and procedures, the analysis of how the system is designed to respond in 
both safe and unsafe situations; and (3) operational scenario, the analysis of the 
environment in which the system is expected to operate. Integrating models that 
quantify each one of these three elements create an analysis capability that is now 
system wide and responsive to ongoing changes in the definition and requirements 
of the operational concept. 
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Application to Independent Approaches 
on Parallel Runways 

As an application of this methodology, we have considered the problem of simul- 
taneous, but independent approaches of two aircraft on parallel runways. In VMC, 
pilots may accept responsibility for maintaining separation between their aircraft 
by visual means. For approaches conducted during IMC, air traffic control per- 
sonnel are responsible for the separation between the aircraft [1]. The FAA allows 
independent parallel approaches to be carried out in VMC with a runway separa- 
tion minimum of 700 feet. In IMC, independent approaches may be conducted to 
runways spaced at least 4,300 feet apart. This minimum is reduced to 3,400 feet if 
the airport is equipped with the PRM system [2]. 

A study performed by the Boeing Commercial Airplane Group has predicted sig- 
nificant increases in runway capacity per hour if dependent approaches could be 
replaced by independent approaches [3]. Because of capacity increases to be 
gained, it is desirable to reduce the minimum runway separation required for in- 
dependent approaches. 

A variety of projects have been undertaken within the past several years to explore 
alerting systems and cockpit displays for the parallel approach situation 
[4, 5, 6, 7, 8]. Aircraft are more closely spaced during parallel approach than during 
any other phase of flight. The potential exists for an aircraft on either runway to 
deviate off course toward another aircraft on the parallel runway. To increase 
safety, an alerting system is used to warn flight crews of these blundering aircraft. 
The goal of the alerting system is to ensure adequate separation between aircraft 
while allowing parallel approaches to be carried out. With reference to our inte- 
grated safety model in Figure 2-1, these studies represent Interaction-Response 
Models. 


2-3 



Independent Approaches on Parallel Runways Concept 
and Operational Procedures 

Figure 2-3 illustrates the components of a typical IAPR concept. 



The IAPR system takes advantage of advances that have been made in communi- 
cation, navigation, and surveillance technologies. Primary among these is GPS- 
based navigation and digital communications for both surveillance and pilot in- 
formation exchanges (ADS-B). GPS-based navigation, with appropriate augmen- 
tation when needed, will provide much more accurate aircraft position and 
velocity information, reducing the need for large protective bubbles around air- 
craft. The accuracy and speed of the ADS-B surveillance data link system is also 
key to successful implementation of the IAPR concept. 

The assumed operational procedure for IAPR is this: 

♦ On-board GPS system provides accurate, timely positional information of 
own ship. 

♦ Position of own ship is broadcast via ADS-B. 

♦ Positions of other ships are received and processed via ADS-B. 
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♦ Location of own ship relative to runway approach and other ships is proc- 
essed and displayed on CDTI monitor. 

♦ Alerting logic sounds alert according to levels of “encounter” criteria an- 
ticipated. 

♦ Avoidance maneuver is initiated to avoid “near collision” event. 

Lacking any involvement of ground control, the IAPR concept just described rep- 
resents a severe and possibly “worst-case” scenario. It is, however, more manage- 
able from a modeling standpoint for this first application. Certainly, future work 
must include models for ground control interaction with aircraft. 

Independent Approaches and Parallel Runways Analysis 
Framework Overview 

Figure 2-4 illustrates the IAPR analysis framework organized with respect to four 
major components: System Reliability Model, Impact Model, Interaction- 
Response Model, and Derivation of system safety statistics. 

Figure 2-4. IAPR Analysis Framework 



Compared with Figures 2-1 and 2-2, the new feature in Figure 2-4 is the Impact 
Model. The function of the Impact Model is to associate a given system functional 
state from the reliability model with an operational capability of the aircraft and 
pilot. For example, a fully operational aircraft can execute a normal approach. 

The system functional state, fully operational, is associated with the flight capa- 
bility, normal approach. Furthermore, the likelihood of the system functional 
stat e, fully operational, is quantified by the Reliability Model, while (conditional) 
safety metrics for the normal approach are determined from the Interaction- 
Response Model through a simulation process. The resulting system-level safety 


2-5 




statistics are calculated by scaling the conditional safety metrics with the likeli- 
hood of the system functional state as illustrated in Figure 2-2. 

An in-depth examination of each of the four analysis components is presented in 
Chapter 3. 
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Chapter 3 

Independent Approaches on Parallel Runways 
Safety Analysis 

Reliability Model 

Role of the Reliability Model 

The objective of the Reliability Model is to predict the state of the aircraft capa- 
bilities at the start of and during an independent approach. In general, when an 
aircraft lines up for an independent approach, it will have been inflight for several 
hours. Assuming that the aircraft had no failures prior to takeoff, in the time from 
takeoff until the start of the approach, failures of components within the systems 
of the aircraft may have occurred that have reduced its capabilities. The reduced 
capabilities, possibly undetected by the pilot, can affect the performance of the 
aircraft during the approach and result in the aircraft drifting or blundering into 
the path of an aircraft approaching the adjacent runway. Alternately, the compo- 
nent failures during en route flight may prevent an independent approach from 
taking place. Procedural rules may prohibit the pilot from attempting an inde- 
pendent approach if there is a known loss of a specific aircraft capability or, in the 
worst case, failures could have caused the loss of the aircraft. The Reliability 
Model will calculate the probabilities of the reduced capabilities that impact the 
safety of the aircraft when an independent approach is attempted. 

Functional Elements 

The first step in developing the Reliability Model needed for the LAPR system 
safety model is to define the aircraft functions that directly and uniquely impact 
the inputs of the Interaction-Response Model. The functions, or capabilities, of the 
aircraft used in the IAPR system safety model are defined in Table 3-1. These 
functions were developed by reviewing the current status of the development of 
the Airborne Information for Lateral Spacing (AILS) research [6,7] and other re- 
lated documentation [9,10]. However, the function definitions and the system de- 
scription of the IAPR system that is presented in the next subsection are not 
strictly based on the AILS research. The function definitions and the system de- 
scription represent the capabilities and components, respectively, that are likely to 
comprise an IAPR system, since a specification of an AILS system does not yet 
exist. 
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Table 3-1. IAPR System Functional Elements 


Function 

Definition 

IAPR RNP navigation 

The capability to perform conformance monitoring of an air- 
craft’s performance and adherence to its approach path 
(RNP). 

ADS-B/surveillance data 
link 

The capability of an aircraft to broadcast, receive, and proc- 
ess ADS-B information for situational awareness, conflict 
avoidance, and airspace management. 

Collision-alerting avionics 

The capability of an aircraft to predict a probable collision 
with another aircraft during approach and landing and to pro- 
vide timely and reliable alerts so that the pilot can avoid the 
collision (this includes alerting logic, processing, and display 
monitors). 

Guidance and control 

The aggregate of all other aircraft capabilities and support 
subsystems exclusive of the previous three functions (e.g., 
propulsion, flight control, and engine control). 

Pilot 

The capability of the pilot(s) to safely operate the aircraft. 


The function definitions are limited to the capabilities of a single aircraft. The 
IAPR system is an aircraft-based collision-avoidance system, but there may be 
dependencies on systems external to the aircraft that can affect safety. The de- 
pendencies with the aircraft that may be approaching the adjacent runway will be 
accounted for because the same function definitions are applied to the adjacent 
aircraft. The dependencies on systems exclusive of the two aircraft are not in- 
cluded in the Reliability Model. These would include any monitoring and interac- 
tion from the ground controller or interaction with other aircraft in the airport 
area. 


The functions defined in Table 3-1 are the capabilities of the aircraft required for 
an independent approach. The first three functions represent capabilities that need 
to be added to present commercial aircraft to support IAPR. The fourth function, 
guidance and control, represents all the capabilities and systems of the aircraft, 
exclusive of those required for the first three functions, which can affect safety of 
an independent approach. The fifth function isolates the capability the pilot (and 
crew) provides in the safe operation of the aircraft. 

System Description 

The system description that follows defines the reliability characteristics of the 
IAPR system. That is, the system description that will be presented defines the 
individual components that can fail, how they are interconnected, the redundancy 
of the components and subsystem functions, and the redundancy management 
logic. 
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To demonstrate the safety analysis methodology, a low-fidelity description of a 
plausible IAPR system is created. A design for the IAPR system does not exist 
now. So, a system is created providing the functionality expected for an IAPR 
system [6, 7] and includes some degree of fault tolerance. The system description 
constructed is complex enough to demonstrate the application of the safety analy- 
sis methodology, but simple enough so minimal resources would be needed to de- 
velop the Reliability Model. The low-fidelity model does not limit the approach. 

Each system component in the system description is assigned to only one function 
to maintain the independence of the functions. The advantage of maintaining the 
independence of the functions is that it enables the probability of any system state 
to be computed in a simple and direct manner. For example, the probability of the 
system being fully operational, at some time t, is simply the product of the prob- 
abilities of each of the functions being in their fully operational states at time t. 

Figures 3-1 through 3-4 present the block diagrams for the system description. 
These are discussed in the next subsections. However, to comprehend the block 
diagrams, several conventions need to be defined. 

♦ Components shown with broken lines are assigned to another function. 
They are included in the block diagrams of some of the functions to indi- 
cate the interconnection between the components of different functions 
and are not considered one of the components necessary for the function. 

♦ Duplicate blocks indicate dual-redundant components. Dual-redundant 
components are both on-line if functional, but only one is necessary for the 
function to be fully operational. 

♦ The connections between components shown should be understood to in- 
dicate that the connected components are fully cross-strapped. For exam- 
ple, in Figure 3-1 the connection between the navigation processors and 
the navigation displays indicated by the arrow means each of the two navi- 
gation processors is connected to each of the navigation displays. 


IAPR-RNP System 

Figure 3-1 presents the block diagram of the IAPR RNP system. The six compo- 
nents shown with solid lines provide the IAPR RNP function defined in Ta- 
ble 3-1. The Global Positioning System (GPS) receiver and INS provide the 
sensed position of the aircraft. The GPS receiver provides discrete position up- 
dates at fixed intervals in time. The INS data are integrated with the position up- 
dates from the GPS receiver to provide a more frequent position update than can 
be obtained with the GPS receiver alone. The data fusion and the navigation com- 
putation are done in the navigation processor. The navigation displays provide 
flight crews with navigation information and with alerts when navigation con- 
tainment is violated. 
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Figure 3-1. IAPRRNP 
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Table 3-2 presents the operational states of the I APR RNP Navigation functions 
that are pertinent to the IAPR safety model. The IAPR RNP Navigation system is 
fully operational if both the GPS receiver and the INS 1 navigation processor and 
1 navigation displays are functional. The system is degraded if either the GPS or 
INS has failed, the failures are detected and compensated for, and an indication 
has been given to the pilot by the system. The failed-safe state is the state of the 
system when component failures have caused the loss of the IAPR RNP naviga- 
tion function and an indication is provided to the pilot to indicate this capability 
no longer is available. Alternately, the failed-uncovered state represents the loss of 
the function, but an indication is not provided to the pilot to indicate the loss of 
this capability. 


Table 3-2. IAPR RNP Navigation Operational States 


State 

Definition 

Impact 

Fully operational 

TSE (total system error) is less 
than containment limit and no 
alert of loss of RNP capability 

Navigation capability available for 
normal approach; ideal distributions 

Degraded 

Loss of either GPS or INS result- 
ing in a degraded navigation ca- 
pability 

Navigation capability available for 
normal approach; nonideal 
distributions 

Failed safe 

Alert of loss of RNP capability 

No longer able to perform independ- 
ent approaches; approach aborted 

Failed uncovered 

TSE is greater than containment 
limit and no alert of loss of RNP 
capability 

Invalid self-knowledge and 
broadcast of navigation data 
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ADS-B/Surveillance Data Link 

Figure 3-2 shows the block diagram of the ADS-B/Surveillance Data Link system. 
The ADS-B/Surveillance Data Link system transmits the IAPR state variable data 
for the aircraft (which the aircraft performing an independent approach on the ad- 
jacent runway can monitor) and receives the IAPR state variable data from the 
adjacent aircraft. The IAPR state variable data broadcast from the aircraft enables 
the Collision-Alerting Avionics of other aircraft to predict a collision. Conversely, 
the IAPR state variable data the aircraft receives from other aircraft enables it to 
predict a collision with these aircraft. The Attitude Heading Reference System 
(AHRS), GPS receiver, and INS provide the sensor data that make up the IAPR 
state variable data. However, these three sensors provide redundant information, 
and sufficient data are available if two of the three are functional. (Note that the 
GPS receiver and the INS are not included in the ADS-B/Surveillance Data Link 
function, having already been included in the IAPR RNP navigation function.) 

Figure 3-2. ADS-B/Surveillance Data Link 
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Table 3-3 presents the operational states of the ADS-B/Surveillance Data Link 
function. For the ADS-B/Surveillance Data Li nk function to be fully operational, 
one ABS-B processor, one ABS-B display, the modulator and transmitter, the re- 
ceiver and demodulator, and the antenna must be functional. The degraded, failed- 
safe, and failed-uncovered states are defined in Table 3-3. 
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Table 3-3, ADS-B/Surveillance Data Link Operational States 


State 

Definition 

Impact 

Fully operational 

Valid broadcast and reception of 
broadcasts from other aircraft 

Transmit and receive functions 
are fully available 

Degraded 

Unable to receive broadcasts 
from other aircraft and may or 
may not receive alert of capabil- 
ity loss; broadcast capability 
functioning 

Knowledge of other aircraft is in- 
valid but approach is allowed 

Failed safe 

Invalid broadcast and alert of 
capability loss and, possibly 
also, loss of reception capability 
of broadcasts from other aircraft 

No longer able to perform inde- 
pendent approaches; approach 
aborted 

Failed uncovered 

Invalid broadcast and no alert of 
capability loss 

Other aircraft do not receive valid 
surveillance data 


Collision- Alerting Avionics 

The Collision-Alerting Avionics block diagram and operational states are shown 
in Figure 3-3 and Table 3-4, respectively. The Collision- Alerting Avionics is fully 
operational if one alerting processor and one alerting displays are functional. The 
alerting processor receives the position of its own aircraft from the IAPR RNP 
navigation system and the IAPR state variable data from the aircraft approaching 
on the adjacent runway from the ADS-B/Surveillance Data Link system. 

Figure 3-3. Collision-Alerting Avionics 
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Table 3-4. Collision-Alerting Avionics Operational States 


State 

Definition 

Impact 

Fully operational 

Collision-alerting capability func- 
tioning properly 

Alerting capability available for 
normal approach 

Failed safe 

Collision alerting not available 
and alert of capability loss 

No longer able to perform inde- 
pendent approaches; approach 
aborted 

Failed uncovered 

Collision alerting not available 
and no alert of capability loss 

Unable to detect blunders of other 
aircraft but approach is not aborted 


Guidance and Control and Pilot 

Figure 3-4 shows the block diagram of the Guidance and Control and Pilot sys- 
tems. The pilot and crew are included here as the block denoted “Pilot.” The 
Guidance and Control system simply represents all the systems of the aircraft ex- 
clusive of the IAPR RNP navigation, ADS-B/Surveillance Data Link, and Colli- 
sion-Alerting Avionics, which impact safety. The pilot provides inputs to engine 
and flight control to ultimately direct the thrust and flight path of the aircraft. Pro- 
pulsion is provided via the engines. Engine control is provided by the engine 
processor using input from the pilot and engine sensors. Flight control is through 
the control processor, which moves the control surfaces based on inputs from the 
pilots and aircraft state and environment sensors. The alternator and power distri- 
bution units (PDUs) generate and distribute electrical power to all components 
requiring it. 


Figure 3-4. Guidance and Control and Pilot Systems 


To all components 
requiring electrical power 



3-7 


















Table 3-5 presents the operational states of the guidance and control function. The 
guidance and control system is fully operational if one engine sensors, one aircraft 
state and environmental sensors, one engine processor, one control processor, one 
guidance and control displays, both engines, one control surfaces, and one alter- 
nator and PDU are functional. The failed-safe state would result from the covered 
failure of one engine. Any uncovered failures or covered failures that result in the 
system not satisfying the definition of fully operational would place the guidance 
and control function in the failed-uncovered state. 

Table 3-5. Guidance and Control Operational States 


State 

Definition 

Impact 

Fully operational 

All other capabilities and support 
subsystems operational 

Capability is fully available for nor- 
mal approach 

Failed safe 

Loss of sufficient capability and 
knowledge of loss 

No longer able to perform inde- 
pendent approaches; approach 
aborted 

Failed-uncovered 

Loss of sufficient capability and 
no knowledge of loss or inability 
to control aircraft 

Worst-case blunder 


Table 3-6 presents the operational states of the pilot function. The pilot function is 
meant to capture the effect of human error in the safety of an independent ap- 
proach. While an actual model of the reliability of the human in the control of the 
aircraft is beyond the level of work being presented here, the pilot function can 
still be broken down into operational states to demonstrate how the reliability of 
the human is integrated in the safety analysis methodology. 

Table 3-6. Pilot Operational States 


State 

Definition 

Impact 

Fully operational 

Pilot functioning nominally with- 
out any faults 

Alerting capability available for 
normal approach 

Recoverable fault 

Pilot fault has occurred; is possi- 
ble to recover from fault 

No impact prior to approach; air- 
craft blunder after start of approach 

Nonrecoverable 

fault 

Pilot fault has occurred; is not 
possible to recover from fault 

No impact prior to approach; air- 
craft blunder after start of approach 


Models 


A set of Markov Reliability Models are constructed from the system described in 
the System Description subsection. The Markov models are developed in accor- 
dance with the techniques presented in Reliability Modeling Methodology for 
IAPR Safety Analysis [1 1]. A separate model is constructed for each function de- 
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fined in Table 3-1. The input files specifying each Markov model to the Semi- 
Markov Unreliability Range Evaluator (SURE) Reliability Analysis Program [12] 
are included in Appendix A. The input files completely specify the Markov Reli- 
ability Models. 

Appendix A also includes Table A-l, which provides the mapping from the nu- 
merical states of each Markov model to the operational states for each function 
defined in the Tables 3-2 through 3-6, respectively. Each state of each Markov 
model is part of one, and only one, of the operational states of one function. Note 
that in Table A-l, the operation state “Unknown” is added for the 
ADS-B/Surveillance Data Link and guidance and control functions. This state re- 
sults from applying a modeling technique referred to as Model Truncation to re- 
duce the number of states in these Markov models [11]. 

Results and Discussion 

The Markov Reliability Models in the last subsection are used to calculate the 
probabilities of being in the operational states of each of the functions. Table 3-7 
presents the baseline failure rates and coverage probabilities for each of the com- 
ponents identified in the system description for the IAPR system. The failure rates 
and coverage probabilities constitute nearly all the input parameters for the mod- 
els. The only missing input parameter is recovery rate from an intermittent human 
failure for the pilot model. The baseline value for this rate is set at 3.6 * 10 2 re- 
coveries per hour. 

The input parameters used are not from any specific source and are selected with 
the intent to highlight the fidelity of the Markov Reliability Models. Typical val- 
ues of failure rates and coverage probabilities are assigned for the components 
that are likely to comprise the system. The failure and recovery rates for the Pilot 
model are not based on any empirical data. 

Table 3-8 shows the calculated probabilities for the operational states of each 
function. The Markov models are evaluated using version 7.9.8 of the SURE Re- 
liability Analysis Program developed by NASA Langley Research Center [12]. 
The Markov model state probabilities are calculated for 4 and 10 hours. These 
represent two time intervals from aircraft takeoff to the lineup point for an inde- 
pendent approach. 

Note that the results in Table 3-8 are presented as bounds on the probabilities of 
being in the states of each function. The bounds occur from two sources. The first 
source, which affects all of the models, is that the SURE program calculates and 
outputs the bounds of the probability of being in the states of the model 
(numerical approximation error). The second source, which affects just the 
ADS-B/Surveillance Data Link and guidance and control Markov models, is the 
model truncation aggregation technique used to limit the size of these models. 
Model truncation introduces some uncertainty into the predictions [11]. 
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Table 3-7. Baseline Failure Rates and Coverage Probabilities 


Component 

Failure Rate (failures/hour) 

Coverage Probability 

IAPR RNP Navigation 

GPS 

3.0E-5 

0.99 

INS 

1.0E-4 

0.99 

Navigation displays 

2.0E-5 

0.999, 0.99 

Navigation processor 

1 OE-5 

0.99, 0.95 

ADS-B/surveillance data link 

AHRS 

1 .OE-5 

0.99 

ADS-B displays 

2.0E-5 

0.999, 0.99 

ADS-B processor 

1 .OE-5 

0.99, 0.95 

Modulator and transmitter 

5.0E-5 

0.99 

Receiver and demodulator 

5.0E-5 

0.99 

Antenna 

1.0E-6 

1.00 

Collision-alerting logic 

Alerting displays 

2.0E-5 

0.999, 0.99 

Alerting processor 

1 .OE-5 

0.99, 0.95 

Guidance and control 

Engine sensors 

4.0E-5 

0.99 

Engine processor 

1 .OE-5 

0.99 

Engine 

1 .OE-5 

0.999 

Alternator and PDU 

2.0E-5 

0.99 

Guidance and control displays 

2.0E-5 

0.999 

State and environment sensors 

4.0E-5 

0.99 

Control processor 

1 .OE-5 

0.99 

Control surfaces 

5.0E-6 

0.99 

Pilot 

Intermittent human failure 

1.0E-4 

1.00 

Permanent human failure 

1.0E-6 

1.00 


Note: For coverage probabilities entered as two numbers, the first number is the coverage 
probability of first failure in redundant components, and the second number is for second failure in 
the redundant components. 


The probabilities shown in Table 3-8 are used by the Impact Model discussed in 
the following section. However, there are some system probabilities produced by 
the Markov Reliabihty Models that are also of interest. Some component failures 
occurring before the approach bneup can preclude an independent approach. Ta- 
ble 3-9 presents two metrics of interest. The first is the probability that insufficient 
capability is available to attempt an independent approach and the approach is 
aborted by the pilot. This is the probability that one or more of the functions, ex- 
cluding the pilot function, is in its failed-safe operational state. The second metric 
is the probability of a loss of the aircraft before the approach lineup. This is the 
probability of being in the failed-uncovered operational state of the guidance and 
control function. 
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Table 3-8. Probabilities of Operational States 



Probability of being operational state 


At 4 hours 

At 10 hours 

Operational state 

Lower 

bound 

Upper 

bound 

Lower 

bound 

Upper 

bound 

IAPR RNP navigation 

Fully operational 

9.9948E-1 

9.9948E-1 

9.9870E-1 

9.9870E-1 

Degraded 

5.15E-4 

5.15E-4 

1 .29E-3 

1 .29E-3 

Failed safe 

5.49E-8 

5.49E-8 

3.43E-7 

3.43E-7 

Failed uncovered 

6.16E-6 

6.16E-6 

1.54E-5 

1 .54E-5 

ADS-B/surveillance data link 

Fully operational 

9.9959E-1 

9.9960E-1 

9.9899E-1 

9.9899E-1 

Degraded 

2.00E-4 

2.00E-4 

5.00E-4 

5.00E-4 

Failed safe 

2.02E-4 

2.02E-4 

5.05E-4 

5.05E-4 

Failed uncovered 

2.96E-6 

2.96E-6 

7.40E-6 

7.41 E-6 

Collision-alerting logic 

Fully operational 

1 .0000E+0 

1.0000E+0 

1.0000E+0 

1 .0000E+0 

Failed safe 

7.83E-9 

7.83E-9 

4.90E-8 

4.90E-8 

Failed uncovered 

9.60E-7 

9.60E-7 

2.40E-6 

2.40E-6 

Guidance and control 

Fully operational 

9.9991 E-1 

9.9991 E-1 

9.9977E-1 

9.9978E-1 

Failed safe 

7.99E-5 

8.00E-5 

2.00E-4 

2.00E-4 

Failed uncovered 

1 .03E-5 

1.03E-5 

2.60E-5 

2.61 E-5 

Pilot 

Fully operational 

1 .0000E+0 

1.0000E+0 

9.9999E-1 

9.9999E-1 

Recoverable failure 

2.78E-7 

3.59E-7 

2.78E-7 

7.83E-7 

Nonrecoverable failure 

4.00E-6 

4.00E-6 

1.00E-5 

1.00E-5 


Table 3-9. Probabilities of Operational States 



Probability 

Metric 

Upper bound at 4 hours 

Upper bound at 10 hours 

Insufficient capability 
independent approach 

2.82E-4 

7.06E-4 

Loss of aircraft before 
approach lineup 

1.03E-5 

2.61 E-5 
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Impact Model 


From the description of the system Reliability Model given in the previous sec- 
tion, it is useful to think of aircraft and pilot as an integration of five functions; 
IAPR RNP navigation, ADS-B/surveillance data link, collision-alerting logic, 
guidance and control, and pilot. Each function is further characterized by its states 
of health or degradation, namely, fully operational, degraded, failed safe, or failed 
uncovered. Each possible system state is associated with an impact that represents 
a potential reduction in system capabilities. The critical question is addressed 
next. 

How System States Impacts Manifest Themselves During the 
Runway Approach 

The correct answer to this question is complex and requires careful study, data 
analysis, and compilation of information from many expert sources. We have not 
undertaken such an investigation within the scope of this initial task. However, to 
illustrate the safety methodology, we assigned different flight tracks for runway 
approaches as the impact of system functional states. In doing so, we can achieve 
an association between the system functional state probabilities of the Markov 
model and the operational safety metrics generated from the Interaction-Response 
Model. The specific details of the Impact Model presented in this illustration 
should not be taken as confirmed, validated facts; they are not! However, the ob- 
jective of the Impact Model, which is to determine and choose a flight track for 
runway approach that reflects a combined operational capability of the aircraft and 
pilot that is consistent with a given system functional state, remains valid and im- 
portant to the realistic evaluation of system safety. 

Right Tracks for Runway Approaches 

The flight tracks used in our Impact Model come from a set of eight piloted flight 
track templates developed by Rockwell-Collins using a Fokker 70 flight simulator 
[6]. These tracks have been widely used as the set of intruder trajectories for 
testing alerting systems [4, 5, 6, 7,8]. A brief description of each is given in Ta- 
ble 3-10. 

Each track is recorded for three different speeds, 130, 145, 160 knots, and under 
both low- and high-turbulence conditions [6,7,8]. 
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Table 3-10. Flight Tracks for Runway Approaches 


Normal approach/landing 

Aircraft heading is aligned to runway heading 

Blunder of 30° 

Aircraft begins runway approach with 30° head- 
ing turned away from own runway and toward 
other runway 

Blunder of 15° 

Aircraft begins runway approach with 1 5° head- 
ing turned away from own runway and toward 
other runway 

Slow heading change blunder of 10° 

Aircraft begins runway approach and slowly de- 
viates from own runway toward other runway by 
1 0° heading change 

Slow heading change blunder of 5° 

Aircraft begins runway approach and slowly de- 
viates from own runway toward other runway by 
5° heading change 

Constant bank angle blunder of 5° 

Aircraft begins runway approach with 5° bank 
angle deviation 

Fake blunder 

Aircraft turns toward other ship’s runway fol- 
lowed by return to desired approach path with 
less than 1 ,000 feet of lateral deviation 

Overadjust blunder 

Aircraft drifts off course away from own and 
other’s runway, recognizes error, makes an ad- 
justment to return to own flight path, and over- 
shoots toward other ship’s runway with less than 
1 ,000 feet of lateral deviation 


For the Interaction-Response Model used in this study, these flight tracks are 
stored as data files in which a stream of data parametersoposition, heading, bank 
angle, and speedois read every 0.4 seconds. Enhancements to the Interaction- 
Response Model are being made to enable users to adaptively change the flight 
track in response to system functional states or other “real-time” situations. These 
enhancements are discussed in Appendix B. 

Impact Model 

The objective of the Impact Model is to choose a flight track that reflects a com- 
bined operational capability of the aircraft and the pilot that is consistent with a 
given system functional state. For example, 

♦ a fully operational aircraft can execute a normal approach; 

♦ undetected or transient failures could result in unintentional drifting of the 
aircraft from a normal approach, such as the fake or overadjust tracks; 
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♦ degraded navigational capability could result in low-level or slow blun- 
dering such as 5 or 10 degree changes; or 

♦ significant failure of guidance or control capability or significant pilot er- 
ror may result in pronounced blunder behavior of 15 or 30 degree changes. 

The Impact Model mapping used in this application is given in Figure 3-5. 

Figure 3-5. Impact Model 


Flight track 

Markov model 
subvectors 

Probability at t = 4 hours 

Probability at t = 10 hours 

norm Normal approach/ landing 

[Nl ,~S3,A1 ,G1 ,P1] 

9.991 8e-1 

9.9796e-1 

Fake; Aircraft fakes blunder 

toward other ships runway 
Oadj: Aircraft drifts away from ow 
and other's runway, then 
overadjusts 

(N4,-S3 r A1,G1,P1]; 
n[N1,-S3 f A3 1 G1,P1]; 
[Nl ,~S3,A1,G1 ,P2] 

3.65e-6 

9.1e-6 

sb5: Constant 5° bank angle 

fN2.~S3 l A1 t G1,P1]; 



blunder 

ah5; Slow 5° heading 
change blunder 

Slo:Slow10° heading 
change blunder 

[Nl ,~S3 f A1,G1 ,P2] 

1.72e-4 

4.3e-4 


Flight track 

Markov model 
subvectors 

Probability at t = 4 hours 

Probability at t = 10 hours 

bl15: 15° heading blunder 


7.15e-6 

1.80e-5 

bl30: 30° heading blunder 

[N1,~S3,A1,Q3,P1]; 
[Nl , -S3.A1 ,G1 ,P3]; 

7.15e-6 

1 .80e-5 


The notation Nl, Al, Gl, and PI refer to the fully operational state of the naviga- 
tion, alerting, guidance and control, and pilot functions, respectively. The notation 
~S3 means any surveillance state except S3. Likewise, N4 is the failed-uncovered 
navigational state while A3 is the failed-uncovered state for the alerting avionics 
function. State N2 is the degraded navigational state while P2 corresponds to 
“recoverable” pilot error. States G3 and P3 are significant error states in the guid- 
ance and control and pilot submodels, respectively. 

Notice that the “aircraft” is being modeled as a vector of the five functional com- 
ponents. Once this association is defined, the Markov model supplies the prob- 
abilities of the vector components. In this illustration, the resulting probability is 
the product of the component probabilities. However, Markov analysis and mod- 
eling is not constrained to “independent” decomposition. Under conditions where 
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it would be important to model and evaluate functional dependencies, the analysis 
method can accommodate that type of complexity. 

We have evaluated the model for two time periods, 4 hours into flight and 10 
hours into flight. These time periods were chosen solely for illustrative purposes. 
In fact, any time period may be evaluated using the Markov analysis as well as the 
two aircraft being evaluated at different time periods to simulate an independent 
but simultaneous approach of two aircraft having been in flight for different time 
periods. 

The fact that Markov analysis can provide time-tagged probabilities of the aircraft 
system state makes it a superior choice for use in system safety analysis. 

Interaction-Response Model 

Background 

The Interaction-Response Model used in this study was developed at Massachu- 
setts Institute of Technology (MIT) under the direction of Professor James 
Kuchar, Department of Aeronautics and Astronautics during 1995-1996 [5,8], 
Draper Laboratory, Inc. obtained this model in January 1997 to expand its capa- 
bilities for both the IAPR safety analysis as well as other interesting considera- 
tions. Further details of these enhancements can be found in Appendix B. 

The original objectives of MITs project were to develop a model of parallel ap- 
proach scenarios incorporating parameters such as runway configuration, blunder 
characteristics, human response delays, and type and accuracy of information 
available to the alerting system; to develop and evaluate a basis for alerting logic 
(i.e., time to impact or range); and to evaluate alerting thresholds based on a toler- 
able rate of false alarms. 

The performance of the prototype alerting system is evaluated using different ap- 
proach trajectories developed from piloted flight simulation tests at Rockwell- 
Collins [6]. These flight tracks were described above. They include normal ap- 
proaches and 6 categories of blunder trajectories: a slow constant-rate turn at a 5° 
bank angle; heading changes of 5°, 10°, 15°, 30°; and two cases in which the in- 
truder began a blunder but returned to its approach path before crossing the threat- 
ened aircraft’s approach path. Separate trajectory data were available for calm and 
turbulent conditions and at airspeeds of 130, 145, and 160 knots. The same tra- 
jectories are used at three runways spacings (1,700, 2,500, and 3,400 feet) and 
over a series of initial longitudinal spacings (within ±1.5 nautical miles of the 
threatened aircraft) to cover a range of possible encounter situations. A total of 
42,822 simulations using 39 different types of trajectories can be performed for 
the evaluations. 
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In the MIT evaluations, the threatened aircraft, the evader, always follows a nor- 
mal approach path while the intruder follows one of the blunder or normal ap- 
proach paths from the simulation tests. The alerting logic is implemented only on 
the evader. If an alert is issued, the evader performs the specified climbing-turn 
avoidance maneuver. The outcome of each approach is recorded, including ( 1 ) 
whether an alert is generated, (2) whether a collision occurs, and (3) whether an 
alert is deemed necessary. Six mutually exclusive categories listed in Table 3-11 
are used to define the possible outcomes. A collision is defined to occur if separa- 
tion at any point in the approach was less than 500 feet. An alert is considered to 
be necessary if a collision would have occurred without an alert. Thus, for exam- 
ple, an alert in a situation in which separation would have been 501 feet without 
the alert is categorized as unnecessary. Such a definition of unnecessary alert, 
though strict, is required as a specific performance metric. A pilot’s or controller’s 
impression of “unnecessary” is important, but it is more subjective and difficult to 
use analytically. 


Table 3-11. Outcome Categories 


Outcome category 

Alert issued? 

Collision occurred? 

Alert necessary? 

Correct rejection 

No 

No 

No 

Missed detection 

No 

Yes 

Yes 

Unnecessary alert 

Yes 

No 

No 

Induced collision 

Yes 

Yes 

No 

Correct detection 

Yes 

No 

Yes 

Late alert 

Yes 

Yes 

Yes 


From Table 3-11, one can see that if an alert is not issued at any time during a run, 
it is classified as either a “correct rejection” (if a collision did not occur) or as a 
“missed detection” (if a collision did occur). If an alert is issued, the outcome is 
placed in one of four categories. An “unnecessary alert” is a case where the in- 
truder is not on a collision course; an alert is issued anyway, and a collision is still 
avoided. If a collision occurs because of the alert, it is classified as an “induced 
collision.” A “correct detection” occurs when a collision is averted because of an 
alert. Finally, a “late alert” is a case in which an alert is issued, but it is too late to 
prevent a collision. 

In summary, the Interaction-Response Model is a simulation tool providing sig- 
nificant flexibility and timeliness in evaluating very difficult aircraft dynamic be- 
havior and alerting response in encounter situations. It is not meant to replace 
human in the loop (HTTL) evaluations. Indeed, elements of the pilot model in- 
cluding response times and the nature and probability of pilot errors and blunders 
are all best developed from data extracted from HITL evaluations. Thus, the 
simulation approach described here goes hand-in-hand with HITL evaluations. 
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Interaction-Response Model Conditional Safety Statistics 

The outcome categories of Table 3-11 can be combined to yield three safety sta- 
tistics defined as follows: 

Probability of reliable operation = # Correct ref + # Correct det. 

Total # of runs 

[Eq. 3-1] 

Probability of collision = # Mis, det. ± # Ind. col ± # Late alerts 

Total # of runs 

[Eq. 3-2] 


Probability of false alarm - # Unnecessary alerts 

Total # of runs 

[Eq. 3-3] 

These three statistics are generated from the Interaction Response model for each 
pair of flight tracks, and they are conditional safety statistics given the flight track 
simulated. To remove this conditioning, we multiply by the probability of flying 
the approach with this flight track, namely, the Markov probability of the flight 
track acquired from the Impact Model. A numerical example is given in the fol- 
lowing baseline performance. 

Baseline Performance 

Table 3-12 shows the results of evaluating the three safety probabilities from the 
simulation outcome categories. 

Table 3-12. Conditional Safety Statistics 


Flight tracks 
[own ship, other ship] 

Probability of 
reliable operation 

Probability 
of collision 

Probability of 
false alarm 

[norm_145, norm_154] 

1 .0000 

0.0000 

0.0000 

[norm_145, fake_145] 

0.9544 

0.0000 

0.0456 

[norm_145, oadj_145] 

0.9125 

0.0000 

0.0875 

[norm_145, sb5_145] 

0.9960 

0.0000 

0.0040 

[norm_145, sh5_145] 

0.9854 

0.0092 

0.0054 

[norm_145, slo_145] 

0.9872 

0.0091 

0.0037 

[norm_145, bl15_145] 

0.9960 

0.0018 

0.0022 

[norm_145, b!30_145] 

0.9872 

0.0037 

0.0091 
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In this evaluation, eight pairs of flight tracks were evaluated by both the MIT and 
Draper Interaction-Response Models at the 1,700-foot runway spacing. A total of 
183 runs were made for each pair of tracks, and the simulation outcomes were 
nearly identical for both models. 

In each case, the ownship was flying the normal approach at 145 knots. The 
alerting logic used in this baseline evaluation was supplied by NASAs Langely 
Research Center. It is a time-/range-based threshold logic. The criterion that in- 
vokes an evasive maneuver on the part of ownship is that the othership be pre- 
dicted to come within 500 feet of ownship within 1 1 seconds. The pilot response 
time was set at 2 seconds; the evasive maneuver was a 25 g pull-up to 2,000 feet 
per minute (fpm) climb, with a 5 degree roll rate to a 30 degree bank angle, and a 
final heading of 45 degrees. The longitudinal initial condition spacing was incre- 
mented at 100-foot intervals thereby producing the 183 runs for each pair of flight 
tracks. 

Combining Model Outputs: System-Level 
Statistics 

Combined Results 

We now complete the baseline performance example by multiplying the condi- 
tional safety statistics by the probability of flying the approach with a given flight 
track. This probability is obtained from the Markov model. We have evaluated the 
Markov model at both 4 and 10 hours of flight time prior to beginning the runway 
approach. This information is shown in Table 3-13. 

Table 3-13. Combined Results at 1,700-Foot Runway Spacing 

System safety statlstlc(t) = £ Pr(Simulation safety stat. I flight track) x Pr (flight track)(t) 

Flight tracks 


System safety statistics 

Rel. op. (4) 

0.9995 

Collisions (4) = 

3.19E-6 

False alarms (4) = 

2.82E-6 

Rel.op. (10) 

0.9993 

Collisions (10) = 

7.97E-6 

False alarms (10) = 

7.05E-6 


Flight tracks 

Conditional simulation safety statistics 
Rel. op. Collisions False alarms 

Probability flight trk 

T = 4hrs. t = lOhrs 

[norm_145, normal 45] 

1 

0 

0 

9.99E-1 

9.98E-1 

[norm_145, fake_145] 

.9544 

0 

.0456 

3.65E-6 

9.1 E-6 

[normal 45, oadj_145] 

.9125 

0 

.0875 

3.65E-6 

9.1 E-6 

[norm_145, sb5^145] 

.996 

0 

.0040 

1 .72E-4 

4.3E-4 

[norm_145, sh5_145] 

.9854 

.0092 

.0054 

1.72E-4 

4.3E-4 

[norm_145, slo_145] 

.9872 

.0091 

.0037 

1.72E-4 

4.3E-4 

[normal 45, bl15_145] 

.996 

.0018 

.0022 

7.15E-6 

1 .8E-5 

[norm_145, bl30^145] 

.9872 

.0037 

.0091 

7.15E-6 

1.80E-5 
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In addition to the 1,700-foot spacing, we completed a baseline evaluation at both 
2,500-foot and 3,400-foot runway spacing. The three sets of safety statistics are 
given in Table 3-14. 

Table 3-14. Safety Statistics at 1,700-Foot, 2,500-Foot, and 3,400-Foot 

Runway Spacings 

2,500-foot spacing 

System safety statistics 

Rel. op. (4) = 0.999524 

Collisions (4) = 3.160E-6 

False alarms (4) = 1.017E-6 


Rel. op. (10) = 0.999310 

Collisions (10) = 7.901 E-6 

False alarms (10)= 2.544E-6 


3,400-foot spacing 

System safety statistics 

Rel. op. (4) = 0.999535 

Collisions (4) = 7.179E-7 

False alarms (4) = 1.013E-6 


Rel. op. (10) = 0.999339 

Collisions (10) = 1.796E-6 

False alarms (10) = 2.533E-6 


1 ,700-foot spacing 

System safety statistics 

Rel. op. (4) = 0.999531 

Collisions (4) = 3.187E-6 

False alarms (4) = 2.81 9E-6 


Rel. op. (10) = 0.999329 

Collisions (10) = 7.968E-6 

False alarms (10) = 7.047E-6 


As runway spacing changes, only the conditional safety statistics change in re- 
sponse; the scaling probabilities from the Markov model remain the same. Al- 
though the actual numerical values should be considered “artificial” and devised 
for the purposes of this example, the trend of the data is reasonable and what one 
would expect. As the time in flight increases prior to runway approach, the overall 
hazard increases and reliable operation decreases. As the runway spacing between 
aircraft increases, the probabilities of collision and false alarm decrease while re- 
liable operation increases. 

In order to demonstrate the approach, we have employed simple models. How- 
ever, the approach is one wherein models can be appropriately tailored for the 
level of detail available or desired. 

We conclude this chapter with an example of sensitivity analysis to show how this 
safety methodology can be used to suggest and evaluate design changes leading to 
improved system performance. 

Sensitivity Analysis: An Example 

The results of the integrated safety analysis can be used to determine how sensi- 
tive the safety statistics are to features of the system architecture, rule and operat- 
ing procedures, or operational scenarios and environment. By understanding these 
sensitivities, design improvements can be proposed and evaluated with a cost/ 
benefit tradeoff analysis. But the first step is to isolate the sensitivity. 
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Referring back to Table 3-13, observe that the slow heading change blunders of 5 
and 10 degrees have the highest collision probabilities, 0.0092 and 0.0091, re- 
spectively. In addition, these tracks have the largest probabilities of occurrence 
with a value of 1.72E-04 at 4 hours and 4.3E-04 at 10 hours. Tracing back to the 
Impact Model and the Reliability Model, we find that the degraded navigation 
state, N2, is the major contributor to these probabilities of occurrence. 

Suppose it were possible to acquire a new, upgraded INS component with a re- 
duced failure rate from IE-04 down to IE-05. Replacing the “old” INS component 
with the new improved element would result in reduced probabilities of occur- 
rence for the slow 5 and 10 degree heading blunders, namely, 5.3E-05 at 4 hours 
and 1.32E-04 at 10 hours. 

Reevaluating the system statistics now yields the following improvements in col- 
lision and false alarm probabilities shown in Table 3-15. 

Table 3-15. Comparison of Results for Improved INS 


Original INS 

Improved INS 

Collisions (4) = 3.19E-06 

Collisions (4) = 1 .01 E-06 

False alarms (4) = 2.82E-06 

False alarms (4) = 1 .02E-06 

Collisions (10) = 7.97E-06 

Collisions (10) = 2.515E-06 

False alarms (10) = 7.05E-06 

False alarms (10) = 2.54E-06 


Alternatively, a rules and procedures change could be made whereby independent 
parallel landings would be precluded when the aircraft is in the degraded naviga- 
tion state, N2. Costs and benefits would have to be evaluated for both the archi- 
tecture option and rules/procedures option to arrive at the best course of action to 
take to improve the overall system performance and reduce the liability of acci- 
dent and false alarm. In either case, the integrated safety analysis can be exercised 
interactively and iteratively in order to arrive at the best solution. 
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Chapter 4 

Conclusions 


Summary of Significant Results 

We have demonstrated an approach to integrating reliability, performance, and 
operational procedures modeling into a system safety analysis. Our methodology 
is distinguished by its ability to merge system design/functionality information 
with the dynamic parameterization of a system’s situation in order to measure ac- 
cident statistics and reliable system operation. In addition, it can perform sensitiv- 
ity analyses to identify weak points in the system’s operation and design. 

Our approach to system safety analysis results from the integration of the Reli- 
ability Model and the Interaction-Response Model. The Interaction-Response 
Model provides information regarding the frequency of encounters and the pre- 
dicted outcome of those encounters as a function of the system’s alerting system 
and ability to resolve encounters. The Reliability Model provides, as a function of 
time, probabilities associated with the critical systems’ availability and failure 
states. Scaling the operations safety metrics from the Interaction-Response Model 
by the system-state probabilities from the Reliability Model creates system-level 
safety statistics. 

Products of this analysis include (1) predicted incident (encounter) statistics; 

(2) predicted accident statistics; and (3) predicted false alarm statistics, as well as 
system availability and reliability. 

As an application of this methodology, we have considered the problem of simul- 
taneous, but independent approaches of two aircraft on parallel runways 
(independent approaches on parallel runways, or IAPR). 

A variety of projects have been undertaken within the past several years to explore 
alerting systems and cockpit displays for the parallel approach situation. Aircraft 
are more closely spaced during parallel approach than during any other phase of 
flight. The potential exists for an aircraft on either runway to deviate off course 
toward another aircraft on the parallel runway. To increase safety, an alerting sys- 
tem is used to warn flight crews of these blundering aircraft. The goal of the 
alerting system is to ensure adequate separation between aircraft while allowing 
parallel approaches to be carried out. With reference to our integrated safety 
model, these studies represent interaction-response models. 

The major limitation of statistical information generated exclusively from an In- 
teraction Response model is that it represents conditional safety statistics given 
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the flight track simulated. To remove this conditioning, we have shown how to 
apply the probability of flying the approach with this flight track using Markov 
analysis to compute this probability. The results give system-level safety statistics 
that can be used to answer important questions such as the variation of reliable 
operation, accidents, and false alarms as a function of different runway spacings. 

The results of the integrated safety analysis can be used to determine how the 
safety statistics are sensitive to features of the system architecture, rule and oper- 
ating procedures, or operational scenarios and environment. By understanding 
these sensitivities, design improvements can be proposed and evaluated with a 
cost/benefit tradeoff analysis. 

Areas for Future Work 

In order to demonstrate the integrated safety analysis methodology, we have em- 
ployed simple models. We believe the approach is one wherein models can be ap- 
propriately tailored for the level of detail either available or desired. Here are 
several areas of future work in which greater model resolution is desired to more 
accurately predict the safety of the air transport concept. 

Pilot Behavior 

The issue of how often a pilot chooses to ignore or override alerting system 
warnings or ground control instructions is certainly important to the safety as- 
sessment of any air transport concept. These elements of the pilot model as well as 
other information relating to response times and the nature and probability of pilot 
errors and blunders are all best developed from data extracted from HITL evalua- 
tions and must continue to be incorporated in the simulation model. 

Ground Controller Behavior and Interaction 

Realistically speaking, there is no emerging air transport concept that will be fully 
implemented without progressing through scheduled participation with current 
day ground control. Certainly, future work must include models for ground con- 
trol interaction with aircraft. 

Environmental Phenomena 

The two-aircraft assumption in our model is certainly a simplification that must be 
removed. Other aircraft in the near vicinity of two aircraft on parallel runways will 
make the issue of evasive maneuvers and resulting “go-arounds” a major consid- 
eration for overall safety. In addition, the forces of wake vortex and environmental 
and structural obstacles must be included to account for safe approaches as well as 
safe evasive maneuvers. 
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Conclusions 


Improved Modeling of the Impact of System Failures and/or Pilot 
Errors on Flight Trajectory 

This issue will require careful study, data analysis, and compilation of information 
from many expert sources. We did not undertake such an investigation within the 
scope of this initial task. To illustrate the safety methodology, we assigned differ- 
ent flight tracks for runway approaches as the impact of system functional states. 
In doing so, we achieved an association between the system functional state prob- 
abilities of the Markov model and the operational safety metrics generated from 
the interaction-response model. The specific details of the Impact Model pre- 
sented in this illustration should not be taken as confirmed, validated facts; they 
are not! However, the objective of the Impact Model, which is to determine and 
choose a flight track for runway approach that reflects a combined operational ca- 
pability of the aircraft and pilot that is consistent with a given system functional 
state, remains valid and important to the realistic evaluation of system safety. 

Desired Capabilities for the Interaction-Response Model 

Draper has continued to enhance the capabilities of the MIT Interaction-Response 
Model. Our current model is completely symmetrized with respect to the capabil- 
ity of ownship and othership. Either aircraft can be assigned any flight track, and 
each aircraft has an alerting system and is capable of evasive maneuver. These 
features are clearly desirable in order to realistically simulate the behavior of two 
aircraft performing independent, parallel approaches. We have discovered some 
interesting consequences of full-dual capability; these are described in Appendix 
B and warrant future investigation. 
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Appendix A 

Reliability Model and Markov Analysis Information 


This appendix lists the input files specifying the Markov Reliability Models to the Semi- 
Markov Unreliability Range Evaluator (SURE) Reliability Analysis program. Table A-l 
provides the mapping from the numerical states from each model to the operational 
states for each function defined in Table A-l. 


Table A-l. Reliability Model to Function State Mapping 


Operational state of function 

Reliability model states 

IAPR RNP navigation 

Fully operational 

1,5. 6,12 

Degraded 

2,4, 8, 9, 10, 11, 13, 14 

Failed safe 

7 

Failed uncovered 

3 

ADS-B/Surveillance data link 

Fully operational 

1,2, 5, 6, 7, 9 

Degraded 

8, 10 

Failed safe 

4 

Failed uncovered 

3 

Unknown (aggregate trapping state) 

11 

Collision alerting avionics 

Fully operational 

1,2, 4,6 

Failed safe 

5 

Failed uncovered 

3 

Guidance and control 

Fully operational 

1 ( 2 f 4, 6, 7, 8, 9, 10, 11 

Failed safe 

5 

Failed uncovered 

3 

Unknown (aggregate trapping state) 

12 

Human factors 

Fully operational 

1 

Recoverable fault 

2 

Non-recoverable fault 

3 
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APR RNP Navigation 


(* IAPR RNP Navigation Reliability Model *) 


(* Failure Rates (failures/hour) *) 


l_g = 3 . 0 e - 5 ; (* GPS *) 
l_i = 1.0e-4; (* INS *) 

l_nd = 2.0e-5; {* Navigation Displays *) 

l_np = 1.0e-5; (* Navigation Processor *) 

(* Coverage Probabilities *) 


c_g = 

c_i = 

c_nd2 

c_nd 

c_np2 

c_np 


0.99; 

0.99; 

= 0.999; 

0.99; 

= 0.99; 
0.95; (* 


(* GPS *) 

<* INS *) 

(* Navigation Displays, two on-line *) 
(* Navigation Displays, one on-line *) 
{* Navigation Processor, two on-line *) 
Navigation Processor, one on-line *) 


(* Transition Rates *) 


1.2 = l_g*c_g; 

1.3 = l_g*(l-c_g) + l_i*(l-c_i) + 2*l_nd* ( l-c_nd2 ) + 2*l_np* {l-c_np2 ) 

1.4 = l_i*c_i; 

1.5 = 2 *l_nd*c_nd2 ; 

1.6 = 2 * l_np *c_np2 ; 

2.3 = l_i*(l-c_i) + 2*l_nd* (l-c_nd2) + 2 *l_np* ( l-c_np2 ) ; 

2.7 = l_i*c_i; 

2.8 = 2*l_nd*c_nd2; 

2.9 = 2*l_np*c_np2 ; 

4.3 = l_g*(l-c_g) + 2 *l_nd* ( 1 -c_nd2 ) + 2 *l_np* ( l-c_np2 ) ; 

4.7 = l_g*c_g; 

4.10 = 2*l__nd*c_nd2 ; 

4.11 = 2 *l_np*c_np2 ; 

5.3 = l_g* ( l-c_g) + l_i*(l-c_i) + l_nd* ( l-c_nd) + 2*l_np* ( l-c_np2 ) ; 

5.7 = l_nd*c_nd; 

5.8 = l_g*c_g; 

5.10 = l_i*c_i; 

5.12 = 2*l_np*c_np2 ; 

6.3 = l_g*{l-c_g) + l_i*(l-c_i) + 2 *l_nd* ( l-c_nd2 ) + l_np* ( l-c_np) ; 

6.7 = l_np*c_np; 

6.9 = l_g*c_g; 

6.11 = l_i*c_i ; 

6.12 = 2 * l_nd * c_nd2 ; 

8.3 = l_i*(l-c_i) + l_nd* ( l-c_nd) + 2 *l_np* ( l-c_np2 ) ; 

8.7 = l_i*c_i + l_nd*c_nd; 

8.13 = 2*l_np*c_np2 ; 

9.3 = l_i* ( l-c_i ) + 2 * l_nd* ( 1 - c_nd2 ) + l_np* ( l-c_np) ; 

9.7 = l_i*c_i + l_np*c_np; 

9.13 = 2 * l_nd*c_nd2 ; 

10.3 = l_g*(l-c_g) + l_nd* (l-c_nd) + 2 *l_np* ( l-c_np2 ) ; 

10.7 = l_g*c_g + l_nd*c_nd; 

10.14 = 2 * l_np * c_np2 ; 

11.3 = l_g*(l-c_g) + 2 *l_nd* (l-c_nd2 ) + l_np* ( l-c_np) ; 

11.7 = l_g*c_g + l__np*c_np; 

11.14 = 2 * l_nd* c_nd2 ; 

12.3 = l_g*(l-c_g) + l_i*(l-c_i) + l_nd* ( l-c_nd) + l_np* ( l-c_np) ; 

12.7 = l_nd*c_nd + l_np*c_np; 

12.13 = l_g*c_g; 
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12,14 = l_i*c_i ; 

13.3 = l_i*(l-c_i) + l_nd* (l-c_nd) + l„np* ( l-c_np ) ; 

13.7 = l_i*c_i + l_nd*c_nd + l_np*c_np; 

14.3 = l_g*(l-c_g) + l_nd* { l-c_nd) + l_np* ( l-c_np) ; 

14.7 = l_g*c_g + l_nd*c_nd + l_np*c_np; 

POINTS = 11; 

start = 1; 
time = 10; 
list = 3; 
prune = le-100; 

run nav . out ; 

ADS-B/Surveillance Data Link 


(* ADS-B/Surveillance data Link Reliability Model *) 

(* Failure Rates ( failures/hour } *) 
l_a = 1.0e-5; {* AHRS *) 

l_g “ 0.0e-6; (* GPS; Equal to zero to maintain independence of models *) 

l_i = 0.0e-6; (* INS; Equal to zero to maintain independence of models *) 

l_ad = 2.0e-5; (* ADS-B Displays *) 

l_ap = 1.0e-5; (* ADS-B Processor *) 

l_mt = 5.0e-5; (* Modulator and Transmitter *) 

l_rd = 5.0e-5; (* Receiver and Demodulator *) 

l_an - 1.0e-6; (* Antenna *) 

(* Coverage Probabilities *) 


c_a = 0.99; 

(* 

AHRS *) 


c_g = 1.0; 

(* 

GPS *) 


c_i = 1.0; 

(* 

INS *) 


c_ad2 - 0.999, 

? (* 

ADS-B Displays, 

two on-line *) 

c_ad - 0.99; 

<* 

ADS-B Displays, 

one on-line * ) 

c_ap2 = 0.99; 

(* 

ADS-B Processor, 

two on-line * 

c_ap = 0.95; 

(* 

ADS-B Processor, 

one on-line * 

c_mt = 0.99; 

{* 

Modulator and Transmitter *) 

c_rd = 0.99; 

(* 

Receiver and Demodulator *) 

c_an = 1.0; 

(* 

Antenna * ) 


(* Transition 

Rates 

*) 



1.2 = l_a*c_a; 

1.3 = l_a* ( l-c_a) + l_g*(l-c_g) + l_i*(l-c_i) + 2*l_ad* ( l-c_ad2 ) + 
l_ap* ( l-c_ap2 ) + l_mt* ( l-c_mt ) + l_an* ( l-c_an) ; 

1.4 - l_g*c_g + l_mt*c_mt + l_an*c_an; 

1.5 = l_i*c_i ; 

1.6 = 2 *l_ad*c_ad2 ; 

1.7 = 2 *l_ap*c_ap2 ; 

1.8 = l_rd; 

2.3 = l_g*(l-c_g) + l_i*(l-c_i) + 2*l_ad* (l-c_ad2 ) + 
l_ap* (l-c_ap2 ) + l_mt* (l-c_mt) + l_an* ( l-c_an) ; 

2.4 = l_g*c_g + l_i*c_i + l_mt*c_mt + l_an*c__an; 

2.9 = 2*l_ad*c_ad2 + 2 *l_ap*c_ap2 ; 

2.10 = l_rd; 

5.3 = l_a*(l-c_a) + l_g*(l-c_g) + 2*l_ad* { l-c_ad2 ) + 
l_ap* (l-c_ap2 ) + l_mt* (l-c_mt ) + l_an* (l-c_an) ; 

5.4 = l_a*c_a + l_g*c_g + l_mt*c_mt + l_an*c_an; 

5.9 = 2*l_ad*c_ad2 + 2 *l_ap*c_ap2 ; 

5.10 = l_rd; 

6,3 = l_a*(l-c_a) + l_g*{l-c_g) + l_i*{l-c_i) + l_ad* ( l-c_ad) + 

2 * l_ap* ( l-c_ap2 ) + l_mt* ( l-c_mt ) + l_an* ( l-c_an) ; 
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6,4 = l_g*c_g + l_ad*c_ad + l_mt *c_mt + l_an*c_an; 

6.9 = l_a*c_a + l_i*c_i + 2 *l_ap*c_ap2 ; 

6.10 = l_rd; 

7.3 = l_a*{l-c_a) + l_g*{l-c_g) + l_i*(l-c_i) + 2 *l_ad* (l-c_ad2 ) + 
l_ap* { l-c_ap) + l_mt * ( l-c_mt ) + l_an* ( l-c_an) ; 

7.4 = l_g*c_g + l_ap*c_ap + l_mt*c_mt + l_an*c_an; 

7.9 = l_a*c_a + l_i*c_i + 2 *l_ad*c_ad2 ; 

7.10 = l_rd; 

8.3 = l_a*(l-c_a) + l_g*(l-c_g) + l_i*(l-c_i) + 2 *l_ad* ( l-c_ad2 ) + 
2 *l_ap* ( l-c_ap2 ) + l_mt * ( l-c_mt ) + l_an* ( l-c_an) ; 

8.4 = l_g*c_g + l_mt *c_mt + l_an*c_an; 

8.10 = l_a*c_a + l_i*c_i + 2*l_ad*c_ad2 + 2 *l_ap*c_ap2 ; 

9.11 = l_a + l_g + l_i + 2*l_ad + 2*l_ap + l_mt + l_rd + l_an; 

10.11 = l_a + l_g + l_i + 2*l_ad + 2*l_ap + l_mt + l_rd + l__an; 

POINTS = 11; 

start = 1; 
time = 10; 
list = 3; 
prune = le-100; 

run sur . out ; 

Collision- Alerting Avionics 


(* Collision Alerting Avionics Reliability Model *) 

(* Failure Rates (failures/hour) *) 

l_nd = 2.0e-5; (* Alerting Displays *) 

l_np = 1.0e-5; (* Alerting Processor *) 

(* Coverage Probabilities *) 

c_nd2 = 0.999; (* Alerting Displays, two on-line *) 

c_nd =0.99; (* Alerting Displays, one on-line *) 

c_np2 = 0.99; ( * Alerting Processor, two on-line *) 

c_np = 0.95; (* Alerting Processor, one on-line *) 

{* Transition Rates *} 

1.2 = 2 * l_nd* c_nd2 ; 

1.3 = 2 * l_nd* ( 1 - c_nd2 ) + 2*l_np* ( l-c_np2 ) ; 

1.4 = 2 *l_np*c_np2 ; 

2.3 = l_nd* ( l-c_nd) + 2*l_np* (l-c_np2 ) ; 

2.5 = l_nd*c_nd; 

2.6 = 2 * l_np * c_np2 ; 

4.3 = 2*l_nd* ( l-c_nd) + l_np* ( l-c_np) ; 

4.5 = l_np*c_np; 

4.6 = 2*l_nd*c_nd2; 

6.3 = l_nd* ( l-c_nd) + l_np* ( l-c__np) ; 

6,5 = l_nd*c_nd + l_np*c_np; 


POINTS = 11; 

start = 1; 
t ime = 10; 
list = 3; 
prune = le-100; 

run alert. out; 
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Reliability Model and Markov Analysis Information 


(* Guidance and Control Reliability Model *) 

(* Failure Rates ( failures /hour ) *) 

l_es = 4.0e-5; (* Engine Sensors *) 

l_ep = 1.0e-5; (* Engine Processor *) 

l_e = 1.0e-5; (* Engine *) 
l_ap = 2.0e-5; (* Alternator and PDU *) 

l_d - 2.0e-5; ( * G & C Displays *) 

l_as = 4.0e-5; (* Aircraft State and Environment Sensors *) 

l_cp = 1.0e-5; (* Control Processor *) 

l_cs - 5.0e-6; (* Control Surfaces *) 

(* Coverage Probabilities *) 

c_es2 =0.99; (* Engine Sensors, two on-line *) 

c_ep2 =0.99; (* Engine Processor, two on-line *) 

c_e = 0.999; (* Engine *) 

c_ap =0.99; (* Alternator and PDU *) 

c_d2 = 0.999; ( * G & C Displays, two on-line *) 

c_as2 = 0.99; (* Aircraft State and Environment Sensors, two on-line *) 
c_cp2 =0.99; (* Control Processor, two on-line *) 

c__cs2 =0.99; {* Control Surfaces, two on-line *) 

(* Transition Rates *) 

1.2 = 2*l_es*c_es2 ; 

1.3 = 2*l_es* (l-c_es2) + 2 *l_ep* ( l-c_ep2 ) + 2*l_e* (l-c_e) + 2 *l_ap* ( l-c_ap) + 
2 * l_d* ( 1 - c_d2 ) + 2*l_as* (l-c_as2) + 2 *l_cp* ( l-c_cp2 ) + 2*l_cs* ( l-c_cs2 ) ; 

1.4 = 2*l_ep*c_ep2 ; 

1.5 = 2 *l_e*c_e ; 

1.6 = 2*l_ap*c_ap; 

1.7 = 2*l_d*c_d2; 

1.8 = 2 *l_as*c_as2 ; 

1.9 = 2*l_cp*c_cp2 ; 

1.10 = 2 *l_cs *c_cs2 ; 

2.3 = l_es + 2*l_ep* (l-c_ep2 ) + 2*l_e* ( l-c_e) + 2 *l_ap* ( l-c_ap) + 

2 * l_d* ( 1 -c_d2 ) + 2*l_as* (l-c_as2) + 2 *l_cp* ( l-c_cp2 ) + 2*l_cs* ( l-c_cs2 ) ; 

2.5 = 2*l_e*c_e; 

2.11 = 2*l_ep*c_ep2 + 2*l_ap*c_ap + 

2*l_d*c_d2 + 2*l_as*c_as2 + 2*l_cp*c_cp2 + 2*l_cs*c_cs2 ; 

4.3 = 2 * l_es * ( l-c_es2 ) + l_ep + 2*l_e* (l-c_e) + 2 *l_ap* ( l-c_ap) + 

2 *l_d* ( l-c_d2 ) + 2*l_as* ( l-c_as2 ) + 2*l_cp* ( l-c_cp2 ) + 2*l_cs* ( l-c_cs2 ) ; 

4.5 = 2 * l_e * c_e ; 

4.11 = 2*l_es*c_es2 + 2*l_ap*c_ap + 

2*l_d*c_d2 + 2*l_as*c_as2 + 2 *l_cp*c___cp2 + 2*l_cs*c_cs2 ; 

6.3 = 2*l_es* (l-c_es2 ) + 2 *l_ep* ( l-c_ep2 ) + l_e*(l-c_e) + l_e + l_ap + 

2 *l_d* ( l-c_d2 ) + 2 *l_as * ( l-c_as2 ) + 2*l_cp* (l-c_cp2 ) + 2 *l_cs* ( l-c_cs2 ) ; 

6.5 = l_e*c_e; 

6.11 = 2 *l_es*c_es2 + 2*l_ep*c_ep2 + 

2*l_d*c_d2 + 2*l_as*c_as2 + 2*l_cp*c_cp2 + 2*l_cs*c_cs2 ; 

7.3 = 2*l_es* (l-c_es2 ) + 2 *l_ep* ( l-c_ep2 ) + 2 *l_e* ( l-c_e) + 2 *l_ap* (l-c_ap) + 
l__d + 2*l_as* {l-c_as2 ) + 2*l_cp* ( l-c_cp2 ) + 2 *l_cs* ( l-c_cs2 ) ; 

7.5 = 2*l_e*c_e ; 

7.11 = 2*l_es*c_es2 + 2*l_ep*c_ep2 + 2*l__ap*c_ap + 

2 *l__as *c_as2 + 2*l_cp*c_cp2 + 2 *l_cs*c_cs2 ; 

8.3 = 2 *l_es* ( l-c_es2 ) + 2*l_ep* ( l-c_ep2 ) + 2 *l_e* ( l-c_e) + 2 *l_ap* ( l-c_ap) + 
2 *l_d* ( l-c_d2 ) + l_as + 2*l_cp* ( l-c_cp2 ) + 2 *l_cs * ( l-c_cs2 ) ; 

8.5 = 2 * l_e * c_e ; 

8.11 = 2*l_es*c_es2 + 2*l_ep*c_ep2 + 2*l_ap*c_ap + 

2*l_d*c_d2 + 2*l_cp*c_cp2 + 2 *l_cs*c_cs2 ; 

9.3 = 2*l_es* ( l-c_es2 ) + 2 *l_ep* ( l-c_ep2 ) + 2 *l_e* ( l-c_e) + 2*l_ap* ( l-c_ap) + 
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2 * l_d* { 1 - c_d2 ) + 2*l_as* (l-c_as2) + l_cp + 2 *l_cs* ( l-c_cs2 ) ; 

9.5 = 2*l_e*c_e; 

9,11 = 2*l_es*c_es2 + 2 *l_ep*c_ep2 + 2*l_ap*c_ap + 

2 * l_d*c_d2 + 2 *l_as*c_as2 + 2 *l_cs*c_cs2 ; 

10,3 = 2*l_es* ( l-c_es2 ) + 2*l_ep* ( l-c_ep2 ) + 2 *l_e* ( l-c_e ) + 2*l_ap* ( l-c_ap 

2 *i_d* ( l-c_d2 ) + 2*l_as* (l-c_as2 ) + 2 *l_cp* ( l-c_cp2 ) + l_cs; 

10.5 = 2 *l_e*c_e ; 

10,11= 2*l_es*c_es2 + 2*l_ep*c_ep2 + 2*l_ap*c_ap + 

2*l_d*c_d2 + 2*l_as*c_as2 + 2 *l_cp*c_cp2 ; 

11,12= 2*l_es + 2 * l_ep + 2*l_e + 2*l_ap + 

2 *l_d + 2*l_as + 2 * l_cp + 2*l_cs; 

POINTS = 11; 

start = 1; 
t ime = 10; 
list = 3; 
prune = le-100; 

run g_and_c . out ; 


Pilot 


{* Human Factors Reliability Model *) 

(* Failure Rates (failures/hour) *) 

l_in = 1.0e-4; (* Intermittent Human Failure *) 

l_p = 1.0e-6; (* Permanent Human Failure *) 

(* Recovery Rates (recoveries/hour) *) 

r_in = ( 1 . /10 . ) * ( 3 600 . /I . ) ; (* Recovery rate from Intermittent Human 

Failure *) 

(* Transition Rates *) 

1.2 = l_in; 

1.3 = l__p; 

2,1 = r_in ; 

POINTS = 11; 

start = 1; 
t ime = 10; 
list = 3; 
prune = le-100; 
trunc = 100; 

run human_2 . out ; 
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Appendix B 

Draper Enhanced and Modified Interaction- 
Response Model 


Draper has continued to enhance the capabilities of the MIT Interaction-Response 
model. Our current model is completely symmetrized with respect to the capabil- 
ity of ownship and othership. In fact, the “ships” are referred to as “lftship” (left 
ship) and “rgtship” (right ship) to correspond with left and right parallel runways. 
Either aircraft can be assigned any flight track, and each aircraft has an alerting 
system and is capable of evasive maneuver. These features are clearly desirable to 
realistically simulate the behavior of two aircraft performing independent, parallel 
approaches. 

The original MIT model did not allow any user adjustment to the simulation proc- 
essing. We have created a user-friendly, front-end menu that offers the user the 
following options: 

♦ Assign any single flight track to lftship and any single flight track to 
rgtship for simulation run. The lftship is no longer restricted to “normal” 
flight track. The code enhancement ensures that if a blunder track is se- 
lected for the lftship, the lftship blunders toward the rgtship’ s runway. In 
addition, the capability to select a single flight track for either or both 
ships allows for much quicker, efficient testing during code development 
and flight track generation. 

♦ Assign a customized file for flight tracks for either or both ships for the 
simulation run. The user can create a file with any number of flight tracks 
to be processed successively by the simulation. The resulting safety data is 
output to a separate file and may be viewed at any time during the simula- 
tion processing. 

♦ Adjust the alerting system’s update frequency. This feature allows the user 
to either speed up or slow down the rate at which the alerting logic is up- 
dated with aircraft positional data. By exercising this option, the user can 
test for safety sensitivity to the alerting logic update rate. Ideally, the 
alerting logic should be updated at a rate that reduces false alarms without 
penalty of increasing the probability of accidents. 

♦ Adjust the trajectory data of the flight track by inputting desired offset val- 
ues. This feature allows the user to generate a new flight track from one of 
the original eight Rockwell-Collins tracks. 
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♦ Simulate “bogus” information processing by a ship’s alerting logic. For 
example, the user specifies a flight track for lftship such as “fake_145 
but specifies that “norm_145” be fed to rgtship’s alerting logic thereby 
simulating the situation that rgtship thinks lftship is on a normal approach. 
Exercising this feature allows the user to investigate what happens when 
misinformation is passed, such as could be the case with undetected or 
transient failures in the system’s navigational or surveillance processing. 

Draper is continuing to test its version of the Interaction-Response model for a 
variety of flight track combinations as well as alerting logics. With respect to at 
least one alerting system, we have discovered a subset of scenario runs in which 
missed detections occurred consistently. This subset consists of combinations of 
“fake” and “oadj” flight tracks at different speeds. These results are preliminary 
and warrant further investigation. 
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Appendix D 

Abbreviations 


ADS-B 

= 

Automatic Dependent Surveillance-Broadcast 

AHRS 

= 

Attitude Heading Reference System 

AILS 

= 

Airborne Information for Lateral Spacing 

ATM 

~ 

Air Traffic Management 

CDTI 

~ 

cockpit display of traffic information 

DGPS 

~ 

Differential Global Positioning System 

FAA 

= 

Federal Aviation Administration 

fmp 

= 

feet per minute 

GPS 

= 

Global Positioning System 

HITL 

= 

human in the loop 

IAPR 

= 

independent approaches on parallel runways 

IMC 

= 

instrument meteorological conditions 

INS 

= 

Inertial Navigation System 

MIT 

= 

Massachusetts Institute of Technology 

PDU 

= 

power distribution units 

PRM 

s 

Precision Runway Monitor 

RNP 

= 

required navigation performance 

TSE 

= 

total system error 

VMC 

- 

visual meteorological conditions 

SURE 

= 

Semi-Markov Unreliability Range Evaluator 
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